Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models
نویسندگان
چکیده
In this study, we derive algorithms for estimating mixed β-divergences. Such cost functions are useful for Nonnegative Matrix and Tensor Factorization models with a compound Poisson observation model. Compound Poisson is a particular Tweedie model, an important special case of exponential dispersion models characterized by the fact that the variance is proportional to a power function of the mean. There are several well known matrix and tensor factorization algorithms that minimize the β-divergence; these estimate the mean parameter. The probabilistic interpretation gives us more flexibility and robustness by providing us additional tunable parameters such as power and dispersion. Estimation of the power parameter is useful for choosing a suitable divergence and estimation of dispersion is useful for data driven regularization and weighting in collective/coupled factorization of heterogeneous datasets. We present three inference algorithms for both estimating the factors and the additional parameters of the compound Poisson distribution. The methods are evaluated on two applications: modeling symbolic representations for polyphonic music and lyric prediction from audio features. Our conclusion is that the compound poisson based factorization models can be useful for sparse positive data. Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013. JMLR: W&CP volume 28. Copyright 2013 by the author(s).
منابع مشابه
Alpha/Beta Divergences and Tweedie Models
We describe the underlying probabilistic interpretation of alpha and beta divergences. We first show that beta divergences are inherently tied to Tweedie distributions, a particular type of exponential family, known as exponential dispersion models. Starting from the variance function of a Tweedie model, we outline how to get alpha and beta divergences as special cases of Csiszár’s f and Bregma...
متن کاملSelecting β-Divergence for Nonnegative Matrix Factorization by Score Matching
Nonnegative Matrix Factorization (NMF) based on the family of β-divergences has shown to be advantageous in several signal processing and data analysis tasks. However, how to automatically select the best divergence among the family for given data remains unknown. Here we propose a new estimation criterion to resolve the problem of selecting β. Our method inserts the point estimate of factorizi...
متن کاملOn Poisson–Tweedie mixtures
*Correspondence: [email protected] 1Department of Mathematics, Ohio University, Athens, OH, USA Full list of author information is available at the end of the article Abstract Poisson-Tweedie mixtures are the Poisson mixtures for which the mixing measure is generated by those members of the family of Tweedie distributions whose support is non-negative. This class of non-negative integer-valued ...
متن کاملFast Parallel Randomized Algorithm for Nonnegative Matrix Factorization with KL Divergence for Large Sparse Datasets
Nonnegative Matrix Factorization (NMF) with Kullback-Leibler Divergence (NMF-KL) is one of the most significant NMF problems and equivalent to Probabilistic Latent Semantic Indexing (PLSI), which has been successfully applied in many applications. For sparse count data, a Poisson distribution and KL divergence provide sparse models and sparse representation, which describe the random variation ...
متن کامل